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Introduction 


• Government agencies, research institutions, and industry 
are using LIDAR imagery for a range of applications. 

• USGS and NASA Stennis Space Center will collaborate to 
characterize LIDAR data products. 

• A number of products are available with limited insight on 
data characteristics. 

• NASA Stennis LIDAR Characterization Approach 

- Utilize in-flight test range to characterize LIDAR data product 
accuracy 

- Develop LIDAR product characterization range at Stennis 

• LIDAR product characterization is the next phase of the NASA V&V 
range development. 
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LIDAR Characterization Range 
Initial Development Approach 


• Follow Draft National Digital Elevation Program (NDEP) 
Guidelines for Digital Elevation Data (2003) 

• Characterize vertical and horizontal accuracy 

• Primary emphasis, initially, is on vertical accuracy 

- Vertical accuracy of LIDAR products is highly dependent on land 
cover type 
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NDEP Guidelines for Digital 

Accuracy Assessment 


Data 


• NDEP recommends using a minimum of 20 checkpoints (30 preferred) 
in each of the ground cover categories representing the area for which 
digital elevation modeling will be performed. 

• The following ground cover categories are identified as the most 
common: 

- Open terrain 

- Tall weeds and crops 

- Brush lands and low tress 

- Forested areas fully covered by trees 

- Urban areas with dense man-made structures 

• NDEP categories not universally applicable 

• Ground cover categories should be relevant to the application and 
study area 
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NDEP Vertical Accuracy Definitions 


• Fundamental Vertical Accuracy - calculated using only open terrain 
checkpoints 

- This value can be used to equitably evaluate and compare vertical 
accuracy among data sets 

- Calculated at the 95 percent confidence level as a function of vertical 
RMSE 

• Supplemental Vertical Accuracy - calculated for other ground cover 
categories (non open terrain) 

- Calculated non-parametrically as the 95th percentile, since errors will likely 
not follow a normal distribution 

• 95 percent of the errors have absolute values equal to or less than the accuracy 

• Consolidated Vertical Accuracy - utilized when 40 or more checkpoints 
are combined for 2 or more ground cover types, representing both open 
terrain and other categories 

- Calculated non-parametrically as the 95th percentile, since errors will likely 
not follow a normal distribution 

• 95 percent of the errors have absolute values equal to or less than the accuracy 
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Stennis Characterization Range 


• The Stennis characterization range is 
built within the Stennis “Fee Area” 

- Approximately 5 mi. x 5 mi. in size 

- Relatively flat terrain 

• ~14 m change in elevation across site 

- Land cover: 

• Buildings 

• Roads 

• Canals 

• Pine Forests 

• Wetlands 

• Open grass 

- Characterization Targets 

• Geodetic network 

• Concrete edge targets 

• Radiometric targets and atmospheric 
instrumentation 
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Stennis Space Center LIDAR Range 

Development 



• Stennis LIDAR range development began prior to release 
of Draft NDEP Guideline 

- Initial range survey planned according to North Carolina Floodplain 
Mapping Program Issue Paper 37 

• Land cover categories selected intended to represent the 5 
common categories identified by NDEP 

• Some categories were selected, even though they were not 
extensive within Stennis site 
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Stennis Land Cover Category A 


• Bare earth/short grass 

- “short grass” is grass that is regularly mowed 

- all Stennis points are in “short grass” areas 
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Stennis Land Cover Category B 


• High grass/weeds 

- “high grass” may be mowed or burned 1-2 times per year 
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Stennis Land Cover Category C 


• Brush / low trees 

- Mostly pine re-growth 

- Uneven canopy allowed, with 
heights ranging from high grass to 
up to one or two 4-meter trees 
within a 5-meter radius from the 
checkpoint. 
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Stennis Land Cover Category D 


• Fully forested 

- Fairly even canopy, 4 meters or greater in height 

- Mostly pine forest in upper canopy 



Slash Pine with Gallberry Understory 



ASPRS, Denver, May 2004 


11 


'a 


Stennis Land Cover Category E 



• Urban 

- Asphalt and concrete surfaces 
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Stennis LIDAR Range Checkpoint 

Requirements 


• Location 

- Each check point separated from nearby obstacles by a distance 
equivalent to the height of the obstacle plus 5 meters. 

• Allows for < 45° incidence angle plus 5 meters horizontal error. 

- Each check point separated from the boundary between forest and 
another category by the height of the forest canopy plus 5 meters. 

• Terrain Slope 

- In accordance with the NDEP Guidelines (2003), checkpoints were 
selected in areas of flat or uniformly sloping terrain (slopes 20% or 
less) for 5 meters in all directions. 

- Flat terrain was preferred to minimize the effect of horizontal error 
on vertical error estimates. 
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Stennis LIDAR Range Check Point 
Requirements (cont’d.) 


• Number of Points 

- 20 check points in each land cover category, based on NDEP 
Guidelines (2003), except for high grass/ weeds 

- 31 points were collected for the “high grass/weeds” category to 
account for the possibility that some of those points may have been 
recently mown 

• Distribution of Points 

- Check points will be well-distributed within each category, especially 
perpendicular to the direction of flight so that they fall under multiple 
flight lines. (N.C. Floodplain Mapping Program Issue Paper 37) 

- Within Stennis sties, it will not be possible to meet the more 
quantitative NDEP Guidelines (2003) suggestions including that at 
least 20% of checkpoints be in every quadrant of the area of interest 
and that all points be separated by at least 10% of the diagonal 
distance across the area of interest. 

• Fully forested category is possible exception 
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Stennis LIDAR Range Survey Techniques 


• All non-forest check points are surveyed with real-time kinematic (RTK) 
GPS. 

- Each RTK GPS check point is occupied twice according to the procedure 
laid out in the NDEP Guidelines (2003). 

• All forest check points are surveyed with conventional topographic 
survey by total station. 

- The conventional surveys are being anchored with twice-occupied RTK 
GPS reference points or A order monuments. 

- Total station survey of the forest check points determines the vertical 
component by trigonometric leveling. 

- If this does not meet the error budget requirements for a specific LiDAR 
characterization, the vertical component for forest points could be surveyed 
later by differential leveling. 

- A follow-on differential leveling effort is not covered in the estimated budget. 

• The Accuracy z (LE95) of check points set to date is 4.2 cm. The 
Accuracy,. (CE95) for their horizontal error is 1 .1 cm. 
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Survey Status 


SSC LiDAR Check Points Surveyed (FY03) 
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• Completion of surveying is awaiting additional funding 

• Goal is to survey a minimum of 20 points in each land cover 
category 
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Stennis Current and 
Proposed Check Points 


SSC Lidar Range Checkpoints 



Legend 
Cover Type 

o 

A - Bare-Earth, Short Grass 

o 

B - High Grass, Weeds 

• 

C- Brush, Low Trees 

• 

D- Fully Forested 

• 

E- Built-Up 
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Potential Future LIDAR Data 
Characterization Range Deve lopment 


• Methods and targets for characterizing the following 
aspects of LIDAR data products should be explored: 

- Planimetric (X,Y) accuracy 

- Vertical (Z) accuracy 

• Reflectance dependencies or effects 

- Height discrimination 

• Minimum resolvable height separation for multiple return systems 

- Sampling density 

- Image quality of intensity-based imaging systems 

• Edge response 

• Radiometry 

• Signal-to-noise 



ASPRS, Denver, May 2004 


18 


'a 


'a 


Planimetric/Horizontal Accuracy 
of Digital Elevation Data 




Horizontal error is more difficult than vertical error to assess in the final 
elevation product 

- Land surface often lacks distinct topographic features needed (e.g. bare earth 
DEMs) 

- Need products with visible features that have been GPS-located 


• Possible assessment approaches listed by NDEP Guideline (2003) 

- Coordinates of the four corners of rooftops of several buildings are accurately 
surveyed (in addition to ground control points surrounding these buildings) and 
compared with LIDAR calibration flights flown over the calibration area from 
multiple directions. 

- Acquire very high-resolution elevation data where well-defined surface features 
such as narrow stream junctions, small mounds, or depressions can be 
identified. Compare surveyed horizontal coordinates with those derived from 
the high-resolution elevation dataset. 

- Use of intensity data to identify features on the ground and compare their 
surveyed horizontal coordinates with those derived from the LIDAR intensity 
images 

• Corresponding intensity data must be earth-referenced by the same process used for 
the elevation data 
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Potential Ideas for Targets to be Developed 


• Horizontal Accuracy 

- Building edges 

- Pattern of retroreflectors on ground, on structure, or across water body 

- Dark target with reflectors on edge 

• Intensity Imagery 

- Surface model of SSC edge target 

• Dependence on pulse footprint size 

- Wire/grid “tree” or trellis structure with substructure (artificial canopy) 

- Sloped wires 

- Netting at angles 

• Height Discrimination 

- Box structures of various sizes and heights; 

- Large scale stairs such as stadium seating at a ball field 

- Retroreflectors placed on structures of different heights 

- Pattern of retroreflectors on angled grid 
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Buildings - Various Heights 






Uniform Slopes 







Linear Features 



ASPRS, 









Steam Line on the 
Army Facility 






Painted Manhole Covers 



•136 painted/surveyed man-hole covers 
located throughout SSC fee area 

•Paint reflectance nominally 50% 

•Manhole cover diameters range between 
0.6 and 2.9 meters 



•Manhole cover centers have been 
geolocated by GPS to <3 cm horizontal 
accuracy and <6 cm vertical accuracy 
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Edge Features 



Edge and Grayscale Target Array, 
as imaged by QuickBird 
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Short and Tall Grasses, Bare Soil 
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Variety of Forest Cover and Density 





Slash Pine with Waxmyrtle Understory 

-- 
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Riparian Areas 
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Drain with Mixed Pine Hardwood 


Forested Wetland 



Water Features 
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Interesting Features 




Satellite Dishes 




Electric Sub-Station 

1 


Pipe Structure and Stacks 
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Interesting Features 
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